Skip to content

CRUD for automations in the API and UI - #2294

Open
Flix6x wants to merge 21 commits into
feat/2288-schedule-automationsfrom
feat/2288-automations-crud
Open

CRUD for automations in the API and UI#2294
Flix6x wants to merge 21 commits into
feat/2288-schedule-automationsfrom
feat/2288-automations-crud

Conversation

@Flix6x

@Flix6x Flix6x commented Jul 11, 2026

Copy link
Copy Markdown
Member

Description

Automations can now be created, updated and deleted through the API and the UI, not only from the
CLI.

API. [POST] /assets/(id)/automations, [PATCH] /assets/(id)/automations/(automation_id) and
[DELETE] /assets/(id)/automations/(automation_id). They require the same rights as deleting the
asset — account admins and consultants — which matches the automation's own access rules: whoever
may read an asset may read its automations, and whoever may delete it may change them. A PATCH
covers the name, the recurrence, the timezone and the activation status; the parameters are
deliberately not editable, so the sensors an automation involves stay the ones its creator was
checked against.

UI. The asset's Automations page gains a creation modal and per-row actions for those users.

Only sensors the creator can access. An automation administered this way may only involve
sensors its creator can access themselves: read access to the sensors it reads from, and
create-children on the sensors it writes to, which is the permission the API already requires for
recording data on a sensor. A refused request gets a 403 naming the sensor and the action. This
sits behind a flag that the API passes; the CLI creates automations without a user and stays
unrestricted.

Which sensors those are depends on the type:

  • Forecasts — the forecaster reports its own input and output sensors, derived from the very
    config and parameters it will run with, so a regressor that filters on sources counts too even
    though it is a sensor reference rather than a plain sensor.
  • Schedules — the scheduler resolves its flex config first, so sensors inherited from the asset
    tree are included, and the outputs are then taken from the fields that name where results are
    recorded: a device's power sensor, its state of charge, consumption and production sensors, and
    the flex-context's aggregates. Everything else the parameters refer to counts as an input.

Timezone through the API. An automation carries the timezone its recurrence is interpreted in.
Creation and update now accept it, so automations administered through the API or the UI are no
longer stuck on the server's timezone. Changing it rebases the scheduling cursor, exactly as the CLI
does.

Refusals leave nothing behind. The data source holding a forecaster's configuration is set up
only once the automation is allowed, so a refused request adds nothing. The check on where a
forecast may be recorded runs after the access check, so a sensor the caller may not read is refused
as forbidden rather than described as being outside the asset.

Consolidation. Creating, updating and deleting live in flexmeasures/data/services/automations.py,
so the CLI and the API share one implementation rather than each building automations inline.

  • Added changelog item in documentation/changelog.rst

Look & Feel

An account admin gets a New automation button and per-row Edit, Activate /
Deactivate and Delete actions:

The Automations page as an account admin, with a New automation button and Edit, Deactivate and Delete on every row

The same page, for the same asset, as a plain user of the same organisation. The automations and
their details are still readable; nothing that would change them is offered:

The same Automations page as a plain user, showing only the listing and the Details button

Creating one asks for the type, the recurrence and the timezone it is interpreted in, and the
parameters — forecast parameters, or a schedule trigger message:

The New automation modal, with fields for name, type, recurrence, timezone and parameters

How to test

See the manual test walkthrough in the PR comments.

pytest \
  flexmeasures/api/v3_0/tests/test_automations_api.py \
  flexmeasures/api/v3_0/tests/test_automations_api_fresh_db.py \
  flexmeasures/cli/tests/test_automations.py \
  flexmeasures/data/tests/test_automations_fresh_db.py \
  flexmeasures/ui/tests/test_asset_crud.py

Coverage includes a forecast on another organisation's sensor, a forecast whose source-filtered
regressor
is another organisation's sensor, a schedule aggregated onto another organisation's
sensor, and the timezone roundtrip through creation and update. Each was verified to fail without
the check it covers.

Further improvements

  • Which sensors a schedule would be recorded on is worked out by resolving the flex config and then
    reading the fields that name where results go. That agrees with the scheduler's own resolution for
    a device's sensor, consumption and production, and errs towards reporting more rather than
    fewer. Checking outputs against what a scheduler actually returns at run time is tracked in Check the sensors a scheduler actually writes to, instead of predicting them when an automation is created #2421.
  • The parameters of an existing automation cannot be edited. Changing what an automation computes
    means deleting it and creating a new one, which keeps the access check honest but is blunt.

Related items

Closes #2372. Part of the automations story #2334. Stacked on #2293.


Sign-off

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on code under GPL or another incompatible license.

Flix6x and others added 2 commits July 11, 2026 18:44
- New endpoints on assets: POST /automations (create, validating parameters
  by automation type), PATCH /automations/<id> (name, cron string, activation
  status) and DELETE /automations/<id>. Managing automations requires the
  same principals that may delete the asset (account admins and consultants).
- The UI automations page gets a 'New automation' modal and per-row
  (de)activate and delete actions, shown to users with management rights.
- Creation, update and deletion logic (incl. audit log records) moved into
  the automations service, shared by the CLI commands and the API endpoints.

Part of #2288

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX
@BelhsanHmida BelhsanHmida linked an issue Jul 30, 2026 that may be closed by this pull request
Flix6x added 4 commits August 5, 2026 18:59
…elves

Context:
- Review of #2290 asked that automations administered through the UI (and hence
  the API) may only involve sensors the creating user has access to; account
  admin rights on the asset should not grant access to another account's sensors

Change:
- Work out the sensors an automation would read from and write to (forecasts:
  the sensor to forecast plus its regressors, and the sensor to save to;
  schedules: the flex-model's device sensors, and whatever the parameters refer to)
- Require read access to the former and create-children (the permission for
  recording data through the API) on the latter, when creating via the API
- The CLI creates automations without a user, and stays unrestricted

Signed-off-by: F.N. Claessen <felix@seita.nl>
Context:
- The endpoint description now states the sensor access rule

Change:
- Regenerated the specs

Signed-off-by: F.N. Claessen <felix@seita.nl>
Context:
- The sensor access rule for created automations needs regression coverage

Change:
- An account admin creating an automation on another account's sensor gets a 403
  naming that sensor, and no automation is created; the same request on their own
  sensor still succeeds (verified to fail without the check)

Signed-off-by: F.N. Claessen <felix@seita.nl>
Context:
- The sensor access rule is user-facing

Change:
- Documented it in the forecasting feature docs, the changelog entry of #2294
  and the API change log

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Flix6x requested a review from BelhsanHmida August 5, 2026 17:00
Flix6x added 2 commits August 5, 2026 23:33
Context:
- Schedulers hand their results to make_schedule as (sensor, data) pairs, and
  those sensors are not only the flex-model's device sensors: a schedule is also
  recorded on a device's state-of-charge, consumption and production sensors, and
  on the flex-context's aggregate-consumption and aggregate-production sensors

Change:
- Derive a schedule's output sensors from all the fields that name where generated
  data goes, at any depth in the flex-model and flex-context (which schedulers
  deserialize themselves, so their sensor references are still raw)
- Everything else the parameters refer to (e.g. price sensors and the sensors of
  inflexible devices, which may also live on the flex-context) counts as an input

Signed-off-by: F.N. Claessen <felix@seita.nl>
Context:
- The flex-context's aggregate-consumption sensor is written to, so it needs the
  same check as the flex-model's own sensors

Change:
- Posting such an automation gets a 403 that names the sensor and the action
  (verified to fail when only the flex-model's sensors are treated as outputs)

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x

Flix6x commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

@BelhsanHmida — Felix reviewed the automations work and asked for one thing to land here: an automation administered through the UI (and hence the API) may only involve sensors that its creator can access. Could you review these commits, and then take the PR over again?

What we contributed:

  • create_automation now works out which sensors an automation would read from and write to, and requires read access to the former and create-children on the latter — the same permission the API requires for recording data on a sensor. A refused request gets a 403 naming the sensor and the action, through the existing api_message mechanism.
  • This is behind a check_permissions flag that the API passes; the CLI creates automations without a user and stays unrestricted.
  • Forecasts: the sensor to forecast plus its regressors are inputs, the sensor to save to (the same sensor by default) is the output.
  • Schedules: schedulers hand their results to make_schedule as (sensor, data) pairs, so the outputs are not only the flex-model's device sensors — a schedule is also recorded on a device's state-of-charge, consumption and production sensors, and on the flex-context's aggregate-consumption and aggregate-production. We derive the outputs from all of those fields at any depth, since the flex-model and flex-context are deserialized by the scheduler itself and their sensor references are still raw at this point. Everything else the parameters refer to (price sensors, inflexible-device sensors — these can live on the flex-context too) counts as an input.
  • Tests cover both a forecast on another account's sensor and a schedule aggregated onto another account's sensor; both were verified to fail without the check. Docs, changelog and API change log updated.

Two things to be aware of:

  1. This branch is ~108 commits behind its base, so it does not yet have the input_sensors / output_sensors properties we added to DataGenerator in Automations - first roundtrip for forecasts #2290. We kept the sensor resolution inside create_automation rather than duplicating those properties, so syncing the stack should be a small edit rather than a conflict — but the two should be collapsed into one implementation then.
  2. Deriving output sensors statically is an approximation of what a scheduler actually returns at run time. If a scheduler starts writing somewhere else, this check will not know about it, so it is worth revisiting whenever a new output field is added.

🤖 Generated with Claude Code

Brings in schedule automations, the timezone and catch-up work, the sensor links and the review fixes from further down the stack.

The CRUD refactor is kept: the CLI still calls create_automation and update_automation rather than building automations inline,
so the services carry the logic that the base had grown there.
create_automation therefore takes a timezone, and update_automation takes one too and rebases the scheduling cursor
whenever it changes what is due, namely the recurrence, the timezone or reactivation.

Two resolutions went further than picking a side.
The forecast output scope is now validated after the access check rather than before,
so that a sensor the user may not read is refused as forbidden rather than described as being outside the asset.
A test that created a forecast automation without a data generator now passes one, as the base requires forecasts to have one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@BelhsanHmida
BelhsanHmida marked this pull request as ready for review August 10, 2026 23:55
BelhsanHmida and others added 12 commits August 11, 2026 01:08
…olves

A regressor that filters on sources deserializes into a sensor reference rather than a sensor,
which collect_sensors skipped, so such a regressor was left out of the sensors an automation reads from.
The access check is built on that list, so a user could set up an automation reading a sensor they cannot read themselves.

Ask the forecaster instead, as it derives its input and output sensors from the same config and parameters it will run with,
and already resolves sensor references. Schedules keep their own collection, as they have no data generator to ask.
Displaying the sensors involved and checking access to them now share one implementation, so they cannot disagree.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
… API

An automation carries the timezone its cron expression is interpreted in, and the CLI can set and change it,
but the API could do neither, so every automation created through the API or the UI was stuck on the server's timezone.
Both the creation and the update schema now accept a timezone, defaulting to FLEXMEASURES_TIMEZONE on creation.

Also restores the OpenAPI spec's version string, which a regeneration during the merge had replaced with the locally installed version.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
…mation is allowed

Creating an automation looked up or created the data source holding the forecaster configuration before checking
whether the user may involve the sensors at all, so a refused request still added a data source within that request.
Nothing committed in between, so this did not outlive the request, but it relied on that rather than on the order of events.
The data source is now set up after the access check, which makes a refused request leave nothing behind by construction.

Also records what the output sensor field list approximates, namely the sensors a scheduler returns results for at run time,
and therefore how it can drift away from them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>

# Conflicts:
#	flexmeasures/api/v3_0/assets.py
#	flexmeasures/data/schemas/automations.py
#	flexmeasures/data/services/automations.py
Permission failures now identify an inaccessible automation dependency only by the sensor ID supplied in the request. This preserves a useful reference for the caller without confirming private sensor names across organisation boundaries.

Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Automation endpoint tests now use function-scoped fresh database fixtures because they create, update, and delete automations and related sensors. The permission cases also assert that forbidden responses retain the submitted sensor ID without disclosing its private name.

Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
The asset automations view now supplies the canonical IANA timezone choices accepted by the automation schema. Keeping the options server-side ensures the create and edit controls offer the same vocabulary that the API validates.

Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Managers can now choose an IANA timezone when creating an automation and edit its name, recurrence, timezone, and active state from the asset page. New automations default to the asset timezone, while the API remains responsible for validating every submitted value.

Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
The asset page regression test verifies that managers receive create and edit timezone fields, that creation starts from the asset timezone, and that both forms include their selected timezone in the corresponding API request.

Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
The automation CRUD entry now records that recurrence timezones are selectable in the user interface and uses the established organisation terminology for the people allowed to manage them.

Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Schedule sensor discovery now leaves creation-time schema and scheduler errors intact so the CLI and API can render their established validation responses. Stored automation resolution still wraps those failures as unknown dependencies for strict permission checks and lenient displays.

Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
@BelhsanHmida BelhsanHmida mentioned this pull request Aug 12, 2026
3 tasks
@BelhsanHmida

Copy link
Copy Markdown
Contributor

Manual test walkthrough

Covers what CRUD adds over #2290 and #2293: administering automations through the API and the UI,
and the rule that you may only involve sensors you can access yourself. The responses below were
captured against a development database on this branch.

You need two logins to see the interesting half:

  • an account admin (or consultant) on the asset — can create, edit and delete automations
  • a plain user of the same organisation — can read automations, and is offered nothing that
    would change them

The examples use asset 242 (Campus, owned by the Campus Demo organisation) and its power sensor
913.

1. The page gains controls, for the right people

/assets/242/automations

As an account admin the listing gains a New automation button and per-row Edit,
Activate / Deactivate and Delete actions. As a plain user of the same organisation the
listing and its Details are unchanged, and none of those controls appear. Both are shown in the
PR description.

2. Create one from the UI

New automation asks for the name, the type, the recurrence, the timezone it is interpreted in,
and the parameters — forecast parameters for a forecast automation, or a schedule trigger message
for a schedule one. Create a forecast automation on sensor 913, daily at 06:00, in
Europe/Amsterdam, and it appears in the listing with its recurrence in words and its timezone.

3. The same over the API

# create
POST /api/v3_0/assets/242/automations
{
  "name": "Daily campus forecast",
  "type": "forecasts",
  "cronstr": "0 6 * * *",
  "timezone": "Europe/Amsterdam",
  "parameters": {"sensor": 913}
}
# → 201, with the automation, its timezone and its scheduling cursor

# change the recurrence, the timezone and the activation status
PATCH /api/v3_0/assets/242/automations/<N>
{"cronstr": "30 7 * * *", "timezone": "Asia/Seoul", "active": false}
# → 200

# delete
DELETE /api/v3_0/assets/242/automations/<N>
# → 204

The timezone is worth checking specifically: an automation created through the API used to be stuck
on the server's timezone, with only the CLI able to change it. Changing it also rebases the
scheduling cursor, so the automation does not immediately catch up on occurrences that only exist in
the new timezone's past.

Note that parameters cannot be patched. What an automation computes is fixed when it is created,
which is what keeps the access check below meaningful — the sensors it involves stay the ones its
creator was checked against.

4. You may only automate sensors you can access

This is the substance of the PR. As an account admin of one organisation, try to forecast a sensor
belonging to another. Sensor 426 below belongs to a different organisation than asset 242:

POST /api/v3_0/assets/242/automations
{"name": "Refused", "type": "forecasts", "cronstr": "0 6 * * *",
 "parameters": {"sensor": 426}}
HTTP 403
{
  "message": "You cannot set up an automation that would read data from sensor 426, because you cannot read data from it yourself.",
  "result": "Rejected",
  "status": "INVALID_SENDER"
}

A regressor that filters on sources is a sensor reference rather than a plain sensor, and is checked
just the same:

POST /api/v3_0/assets/242/automations
{"name": "Refused", "type": "forecasts", "cronstr": "0 6 * * *",
 "parameters": {"sensor": 913},
 "config": {"regressors": [{"sensor": 426, "source-types": ["forecaster"]}]}}
HTTP 403
{
  "message": "You cannot set up an automation that would read data from sensor 426, because you cannot read data from it yourself.",
  "result": "Rejected",
  "status": "INVALID_SENDER"
}

Three things are worth confirming while you are here:

  • Nothing is created. The automation does not appear in the listing, and no data source is left
    behind for the forecaster it would have used — the data source is set up only once the automation
    is allowed.
  • The message names the id, not the name. Refusing access should not disclose what a sensor you
    cannot read is called.
  • Writing is checked separately from reading. Reading a sensor's data needs read access;
    recording data on one needs the same permission the API requires for recording data by hand.

For schedules the same applies to whatever the schedule would be recorded on, including sensors the
scheduler inherits from the asset tree rather than ones written out in the trigger message.

5. The CLI stays unrestricted

flexmeasures add automation --asset 242 --name "From the CLI" --cron "0 6 * * *" --sensor 913

The CLI runs without a user, so it is trusted and not subject to the check above. That is deliberate:
the restriction is about what a user may set up through the API or the UI.

6. Clean up

Delete anything you created, from the listing's Delete action or with
flexmeasures delete automation --id N --force.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CRUD for automations in the UI

2 participants